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(54) Method of fast automatic exposure or gain control in a MOS image sensor 



(57) In a CMOS image sensor, it is important to con- 
trol the exposure time or gain. For varying changes in 
ambient light, the exposure time of the array must be 
changed. The present invention describes a method for 
controlling the exposure time or gain. Each of the inten- 
sities of the pixels in a frame is analyzed to provide an 



indication of the number of pixels having an intensity 
above or below multiple predetermined thresholds This 
information is then used to change the exposure time 
The magnitude of change in the exposure time is also 
determined by the number of pixels having an intensity 
above or below multiple predetermined thresholds, as 
the case may be. 
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Description 

[0001] The present invention relates generally to 
CMOS image sensors, and more particularly, to a meth- 
od of controlling an exposure time or gain of an image 5 
sensor. 

[0002] Integrated circuit technology has revolution- 
ized various fields including computers, control sys- 
tems, telecommunications, and imaging. For example, 
in the imaging field, CMOS image sensors have proved 
to be less expensive to manufacture relative to CCD im- 
aging devices Further, for certain applications CMOS 
devices are superior in performance The pixel elements 
in a MOS device can be made smaller and therefore pro- 
vide a higher resolution than CCD image sensors. In ad- 
dition, the signal processing logic necessary can be in- 
tegrated alongside the imaging circuitry, thus allowing 
for a single integrated chip to form a complete stand 
alone imaging device. 

[0003] Examples of MOS imaging devices are de- 
tailed in "A % Inch Format 250K Pixel Amplified MOS 
Image Sensor Using CMOS Process" by Kawashima et 
al., IEDM 93-575 (1993), and "A Low Noise Line-Ampli- 
fied MOS Imaging Devices" by Ozaki et aL, IEEE Trans- 
actions off Electron Devices, Vol. 38, No 5, May 1991. 
In addition, U.S Patent No 5,345,266 to Denyer titled 
"Matrix Array Image Sensor Chip" describes a MOS im- 
age sensor. CMOS image sensors are now available 
from a variety of manufacturers, including the assignee 
of the present invention, OmniVision Technologies, Inc. 
[0004] The primary building block of an image formed 
by a CMOS image sensor is a pixel. The number, size 
and spacing of the pixels determine the resolution of the 
image generated by the imaging device The pixels of a 
CMOS image sensor are semiconductor devices that 
transform incident light photons into current signals. The 
signal produced by each pixel is generally extremely 
small. 

[0005] One important parameter that the CMOS im- 
age sensor must be able to control is the exposure time 
of each pixel to incident light. Similar to light exposure 
time for photographic film, the exposure time of each 
pixel must be adjusted to compensate for variations in 
lighting conditions, such as for indoor or outdoor lighting. 
An exposure time that is too long will result in an image 
that is overly bright and washed out In contrast, an ex- 
posure time that is too short will result in an image that 
is dark and difficult to view. 

[0006] Another method of varying the intensity of the 
pixel signals is to adjust the gain of the amplification cir- 
cuitry. In many ways, controlling the gain is substantially 
similar to controlling the exposure time. By varying one 
or both, the image generated by the image sensor can 
be optimized for viewing. 

[0007] U.S. Patent No. 5,734,426 describes one prior 
art method of controlling exposure time. While adequate 
for many applications, the speed at which the exposure 
time can be automatically changed is insufficient for cer- 



tain applications. For example, for automobile applica- 
tions, where the automobile may travel from sunlight, to 
darkness (such as in a tunnel), back to sunlight, it is im- 
portant for the image sensor to be able to adjust its ex- 
posure time quickly. 

[0008] In a first aspect the present invention provides 
a method of automatically controlling the exposure time 
of an image sensor, said image sensor including a ma- 
trix of pixels according to claim 1 . 

Brief Description of the Drawings 

[0009] In the drawings, like reference numerals refer 
to like parts throughout the various views of the non- 
limiting and non-exhaustive embodiments of the present 
invention, and wherein: 

Figure 1 is a schematic diagram of a CMOS image 
sensor. 

Figure 2 is a flow diagram illustrating the method of 
the present invention. 

Detailed Description of the Illustrated Embodiments 

[001 0] In the following description, numerous specific 
details are provided, such as the identification of various 
system components, to provide a thorough understand- 
ing of embodiments of the invention. One skilled in the 
art will recognize, however, that the invention can be 
practiced without one or more of the specific details, or 
with other methods, components, materials, etc. In still 
other instances, well-known structures, materials, or op- 
erations are not shown or described in detail to avoid 
obscuring aspects of various embodiments of the inven- 
tion. 

[0011] Reference throughout this specification to 
"one embodiment" or "an embodiment" means that a 
particular feature, structure, or characteristic described 
in connection with the embodiment is included in at least 
one embodiment of the present invention. Thus, the ap- 
pearance of the phrases "in one embodiment" or "in an 
embodiment" in various places throughout this specifi- 
cation are not necessarily all referring to the same em- 
bodiment. Furthermore, the particular features, struc- 
tures, or characteristics may be combined in any suita- 
ble manner in one or more embodiments. 
[0012] With reference to FIGURE 1, an architecture 
for a CMOS imaging array 101 includes a rectangular 
matrix of pixels 103. The number of pixels in the hori- 
zontal orx-direction, and the number of pixels in the ver- 
tical or y-direction, constitutes the resolution of the im- 
aging array 101. Each of the pixels 103 in a vertical col- 
umn routes its signal to a single charge amplifier 105. 
[0013] The retrieval of information from the pixels 1 03 
follows the well-known raster scanning technique In par- 
ticular, a row of pixels 103 is scanned sequentially from 
left to right. The next row is then scanned in this manner 
until all rows have been scanned sequentially from top 
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to bottom At the end of each complete scan of the entire 
array 101, a vertical blanking period of predetermined 
time occurs until the raster scanning pattern is repeated. 
This type of scanning follows the NTSC scanning 
scheme. Control circuitry of conventional design is op- 
erative to sequentially read the pixels 103 in this man- 
ner. 

[0014] As each pixel is scanned, the signal from that 
pixel is provided to the charge amplifier 105 for that col- 
umn Thus, the charge amplifiers 105 receive signals se- 
quentially The sequential signals from the charge am- 
plifiers 105 are then forwarded to a second-stage am- 
plifier 107, which amplifies the signals so that they may 
be further processed. In various embodiments, either 
the column amplifier or the second-stage amplifier may 
be eliminated. 

[001 5] The present invention uses the magnitudes of 
the amplified signals from each of the pixels to deter- 
mine the exposure time for each pixel. Generally speak- 
ing, the exposure time for each pixel can be varied by 
controlling the raster scan control algorithm. 
[001 6] The method of the present invention is shown 
in Figure 2. While the following describes the method of 
the present invention in connection with controlling ex- 
posure time, the method applies equally well to control- 
ling gain. At box 201, the magnitude of a pixel signal 
from a pixel in the array is determined. Next, at box 203, 
a determination is made as to whether the pixel signal 
is greaterthan a predetermined value VHIGH. Typically, 
the pixel signal will be between 0.3 volts and 1.0 volts. 
This voltage level is set by the NTSC television stand- 
ard. However, it can be appreciated that the magnitude 
of the pixel signal may vary depending upon the partic- 
ular types of pixels being used, the amount of amplifi- 
cation in the column amplifiers or second-stage ampli- 
fiers, and various other factors. Thus, the precise volt- 
ages given herein are simply examples and should not 
be construed as limiting. 

[0017] In one embodiment, a typical value for VHIGH 
is 0.8 volts. If the magnitude of the pixel signal is greater 
than VHIGH, then at box 205 a counter, NW, is incre- 
mented by 1 . After incrementing, the next step is at box 
211. 

[0018] However, if at box 203, the pixel signal is not 
greater than VHIGH, then, at box 207, a determination 
is made as to whether the pixel signal is less than an- 
other predetermined threshold value, VLOW. In one em- 
bodiment, VLOW is 0.5 volts. If the pixel signal is less 
than VLOW, then at box 209, a counter, NB, is incre- 
mented by 1. After box 209, control is provided to box 
211. 

[0019] While the process of boxes 203, 205, 207, and 
209 is being conducted for a pixel signal, the same pixel 
signal is also being analyzed relative to a second set of 
"super thresholds (hence the designation "S" in the 
threshold names) Specifically, at box 202, a determina- 
tion is made as to whether the pixel signal is greaterthan 
a predetermined value VSHIGH. A typical value for 



VSHIGH is 0.9 volts. If the magnitude of the pixel signal 
is greater than VSHIGH, then at box 204 a counter, 
NVW, is incremented by 1. After incrementing, the next 
step is at box 211. 

5 [0020] However, if at box 202, the pixel signal is not 
greater than VSHIGH, then, at box 206, a determination 
is made as to whether the pixel signal is less than an- 
other predetermined threshold value, VSLOW. In one 
embodiment, VSLOW is 0.4 volts. If the pixel signal is 

10 less than VSLOW, then at box 208, a counter, NVB, is 
incremented by 1 . After box 208, control is provided to 
box 211. 

[0021] Next, at box 211, a determination is made as 
to whether or not any more pixels in the frame still need 

15 to be analyzed. If so, the process of boxes 201-209 is 
repeated It should be noted that it is not necessary to 
analyze all of the pixels in a frame. Indeed, a subset of 
the pixels in the frame may be analyzed However, in one 
preferred embodiment, all of the pixels are analyzed. 

20 Thus, if the entire pixel array of the frame has been an- 
alyzed, then at box 213, a determination is made as to 
whether or not the counter NW is greater than a prede- 
termined threshold value KW. Typically, KW is a number 
that is about 25% of the total number of pixels in the 

25 imaging array. 

[0022] If NW is not greater than KW, then there is no 
decreasing of the exposure time, and control goes to 
box 221. However, if NW is greaterthan KW, then there 
is a need to decrease the exposure time. Recall that the 

30 counter NW indicates the number of pixels that are ex- 
hibiting high output signals, corresponding to a large 
amount of impinging light If a large number of pixels ex- 
hibit high output signals, then this indicates that there 
may be an overly long exposure time In such a case, 

35 the exposure time should be decreased. 

[0023] In the prior art, after determining that NW is 
greater than KW, the exposure time is decreased by a 
predetermined decrease increment Nevertheless, if the 
predetermined decrease increment is made too small, 

40 it may take several frames of image before the exposure 
is adequately corrected. However, if the predetermined 
decrease increment is set too high, then there may be 
an 'overshoot" in the amount of decrease in exposure 
time and the video stream may appear "jumpy" 

45 [0024] In any event, in order to improve upon the prior 
art automatic exposure correction technique, in accord- 
ance with the present invention, a second step in the 
analysis takes place to determine an appropriate mag- 
nitude of the decrease to exposure time. Specifically, at 

so. box 215, a determination is made as to whether or not 
the counter NVW is greater than a threshold value KVW. 
If so, then at box 217, the exposure time is decreased 
by a first predetermined decrease increment. In one em- 
bodiment, the first predetermined decrease increment 

55 is defined as a percentage (P1) of the exposure time. 
However, if the counter NVW is not greater than the 
threshold value KVW, then at box 219, the exposure 
time is decreased by a second predetermined decrease 
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increment. In one embodiment, the second predeter- 
mined decrease increment is defined as a percentage 
(P2) of the exposure time Typically, P1 is greater than 
P2, and in one embodiment, P1 is 50 percent and P2 is 
6 percent. 

[0025] Thus, to summarize, if it is determined that a 
large number of pixels are outputting high signal levels 
(NW > KW), then the exposure time should be de- 
creased. A next process takes place to determine the 
magnitude by which the exposure time should be de- 
creased, P 1 or P2. For this analysis, the counter NVW 
(which measures the number of pixels that output a sig- 
nal above a "super" threshold) is used. Specifically, if 
the counter NVW is greater than a threshold number 
KVW, then a larger decrease in the exposure time is 
warranted. If the counter NVW is not greater than KVW, 
then a smaller decrease in the exposure time is used. 
[0026] A similar process is followed for increasing the 
exposure time. Thus, if at box 213, NW is not greater 
than KW, then at box 221, a determination is made as 
to whether or not the counter NB is greater than a pre- 
determined threshold value KB. Typically, KB is a 
number that is about 75 % of the total number of pixels 
in the imaging array. 

[0027] If NB is not greater than KB, then there is no 
increasing of the exposure time However, if NB is great- 
er than KB, then there is a need to increase the exposure 
time. Recall that the counter NB indicates the number 
of pixels that are exhibiting low output signals, corre- 
sponding to a small amount of impinging light. If a large 
number of pixels exhibit low output signals, then this in- 
dicates that there may be an overly short exposure time. 
In such a case, the exposure time should be increased. 
[0028] In the prior art, after determining that NB is 
greater than KB, the exposure time is increased by a 
predetermined increase increment. Nevertheless, if the 
predetermined increase increment is made too small, it 
may take several frames of image before the exposure 
is adequately corrected. However, if the predetermined 
increase increment is set too high, then there may be 
an 'overshoot" in the amount of increase in exposure 
time and the video stream may appear "jumpy". 
[0029] In any event, in order to improve upon the au- 
tomatic exposure correction technique, in accordance 
with the present invention, a second step in the analysis 
takes place to determine an appropriate magnitude of 
the increase to exposure time Specifically, at box 223, 
a determination is made as to whether or not the counter 
NVB is greater than a threshold value KVB. If so, then 
at box 225, the exposure time is increased by a first pre- 
determined increase increment. In one embodiment, the 
first predetermined increase increment is defined as a 
percentage (P3) of the exposure time However, if the 
counter NVB is not greater than the threshold value 
KVB, then at box 227, the exposure time is increased 
by a second predetermined decrease increment In one 
embodiment, the second predetermined decrease in- 
crement is defined as a percentage (P4) of the exposure 



time Typically, P3 is greater than P4, and in one embod- 
iment, P3 is 100 percent and P4 is 6 percent. 
[0030] Thus, to summarize, if it is determined that a 
large number of pixels are outputting low signal levels 
5 (NB > KB), then the exposure time should be increased. 
A next process takes place to determine the magnitude 
by which the exposure time should be increased, P3 or 
P4. For this analysis, the counter NVB (which measures 
the number of pixels that output a signal below a "super" 
10 threshold) is used. Specifically, if the counter NVB is 
greater than a threshold number KVB, then a larger in- 
crease in the exposure time is warranted. If the counter 
NVB is not greater than KVB, then a smaller increase in 
the exposure time is used. 

[0031 ] It can be appreciated that the method of Figure 
2 is typically implemented on the image sensor integrat- 
ed circuit. In particular, most image sensor chips include 
a sensor array and processor circuitry. The processor 
circuitry, in one example, may be programmed to per- 
form the analysis detailed in Figure 2. 
[0032] While the invention is described and illustrated 
here in the context of a limited number of embodiments, 
the invention may be embodied in many forms without 
departing from the spirit of the essential characteristics 
of the invention. The illustrated and described embodi- 
ments are therefore to be considered in ail respects as 
illustrative and not restrictive. For example, the above 
detailed description has been given in terms of adjusting 
the exposure time. However, the techniques described 
above apply equally to gain control. For example, in- 
stead of increasing or decreasing the amount of expo- 
sure, the amount of gain can be similarly increased or 
decreased Still further, both the exposure time and gain 
can be increased or decreased as the need may be 
Thus, the scope of the invention is indicated by the ap- 
pended claims rather than by the foregoing description, 
and all changes which come within the meaning and 
range of equivalency of the claims are intended to be 
embraced therein. 



Claims 

1 . A method of automatically controlling the exposure 
time of an image sensor, said image sensor includ- 
ing a matrix of pixels, said method comprising: 

determining a value N W as the number of pixels 
in said matrix that output a pixel signal greater 
than a value VHIGH; 

determining a value NB as the number of pixels 
in said matrix that output a pixel signal less than 
a value VLOW; 

determining a value NVW as the number of pix- 
els in said matrix that output a pixel signal 
greater than a value VSHIGH; 
determining a value NVB as the number of pix- 
els in said matrix that output a pixel signal less 
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than a value VSLOW, 

determining if NW is greater than a first prede- 
termined high threshold KW, and if so: 

determining if NVW is greater than a sec- 
ond predetermined high threshold KWV, 
and if so, decreasing said exposure time 
by a first decrease percentage P1, but if 
not, decreasing said exposure time by a 
second decrease percentage P2; and 

determining if NB is greater than a first prede- 
termined low threshold KB, and if so: 

determining if NVB is greater than a sec- 
ond predetermined low threshold KVB, and 
if so, increasing said exposure time by a 
first increase percentage P3, but if not, in- 
creasing said exposure time by a second 
increase percentage P4. 

2. The method of Claim 1 wherein VSLOW is less than 
VLOW and further wherein VSHIGH is greater than 
VHIGH. 

3. The method of Claim I wherein said first decrease 
percentage PI is greater than said second decrease 
percentage P2. 

4. The method of Claim 3 wherein P1 is about 50 per- 
cent and P2 is about 6 percent. 

5. The method of Claim I wherein said first increase 
percentage P3 is greater than said second increase 
percentage P4. 

6. The method of Claim 5 wherein P3 is about 1 00 per- 
cent and P4 is about 6 percent. 

7. The method of Claim I wherein said predetermined 
values of KB and KW are about 75% and 25% of 
the total number of pixels in said matrix. 

8. A method of automatically controlling the amplifica- 
tion gain of an image sensor, said image sensor in- 
cluding a matrix of pixels, said method comprising 

determining a value NW as the number of pixels 
in said matrix that output a pixel signal greater 
than a value VHIGH; 

determining a value NB as the number of pixels 
in said matrix that output a pixel signal less than 
a value VLOW; 

determining a value NVW as the number of pix- 
els in said matrix that output a pixel signal 
greater than a value VSHIGH; 
determining a value NVB as the number of pix- 
els in said matrix that output a pixel signal less 



than a value VSLOW, 

determining if NW is greater than a first prede- 
termined high threshold KW, and if so: 

5 determining if NVW is greater than a sec- 

ond predetermined high threshold KVW, 
and if so, decreasing said amplification 
gain by a first decrease percentage P1 , but 
if not, decreasing said amplification gain by 

10 a second decrease percentage P2, and 

determining if NB is greater than a first prede- 
termined low threshold KB, and if so: 

15 determining if NVB is greater than a sec- 

ond predetermined low threshold KVB, and 
if so, increasing said amplification gain by 
a first increase percentage P3, but if not, 
increasing said amplification gain by a sec- 

20 ond increase percentage P4. 

9. The method of Claim 8 wherein VSLOW is less than 
VLOW and further wherein VSHIGH is greater than 
VHIGH. 

25 

10. The method of Claim 8 wherein said first decrease 
percentage P1 is greater than said second de- 
crease percentage P2. 

30 11. The method of Claim 1 0 wherein P1 is about 50 per- 
cent and P2 is about 6 percent 

12. The method of Claim 8 wherein said first increase 
percentage P3 is greater than said second increase 

35 percentage P4. 

13. The method of Claim 12 wherein P3 is about 100 
percent and P4 is about 6 percent. 

40 1 4. The method of Claim 8 wherein said predetermined 
values of KB and KW are about 75% and 25% of 
the total number of pixels in said matrix. 
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